1. Field of the Invention
The present invention relates to an imaging device, and more particularly, to a technique for forming an object image which passes through different regions of an imaging lens in two directions on an imaging element to acquire different viewpoint images.
2. Description of the Related Art
JP2010-145544A, JP2007-279512A, and JP2009-527007T are examples of a pupil-division-type stereoscopic imaging device according to the related art.
JP2010-145544A discloses a stereoscopic imaging device that performs shading correction on an image A and an image B which are obtained from different exit pupil regions to calculate the amount of phase difference. The shading correction is performed by dividing the image B by the image A or multiplying the image A and the image B.
JP2007-279512A discloses a stereoscopic imaging device that shifts a microlens provided on a photodiode to perform stereoscopic imaging. When the stereoscopic imaging is performed, the shift direction of the microlens and the amount of shift of the microlens are changed by the center and periphery of a light receiving surface of a base to reduce the influence of shading on image quality.
For example, JP2009-527007T discloses a single-eye stereoscopic imaging device including an optical system shown in FIG. 13.
In the optical system shown in FIG. 13, an object image which passes through different regions of a main lens 1 and a relay lens 2 in the lateral direction is pupil-divided by a mirror 4 and the divided images are formed on imaging elements 7 and 8 through imaging lenses 5 and 6, respectively.
FIGS. 14A to 14C are diagrams illustrating the separated state of an image formed on imaging elements due to the difference among the front focus, the best focus, and the rear focus. In FIGS. 14A to 14C, the mirror 4 shown in FIG. 13 is not shown in order to compare the difference in separation according to the focus.
As shown in FIG. 14B, among the pupil-divided images, the best focus images are formed (superimposed) at the same position on the imaging element. However, as shown in FIGS. 14A and 14C, the front focus images and the rear focus images are formed (separated) at different positions on the imaging element.
Therefore, when the object images which are pupil-divided in the lateral direction are acquired through the imaging elements 7 and 8, a left viewpoint image and a right viewpoint image with viewpoints which vary depending on the object distance are obtained. A 3D image can be generated from the obtained left viewpoint image and right viewpoint image.
It has been known that, when the image of an object is formed on the imaging surface of the imaging element by an optical system, such as a zoom lens, the image captured by the imaging element is blurred by the influence of the aberration of the optical system, as compared to the original object. The intensity distribution g of the image obtained at that time is calculated by adding noise n to the convolution of the brightness distribution f of the original object and a point spread function h indicating the imaging performance of the optical system, as follows:g=f*h+n(* is convolution).  [Expression A]
Since g, h, and n have been known, the brightness distribution f of the original object can be calculated from Expression A. In this way, a technique for removing the burring of the optical system using signal processing to obtain an ideal image is called the “restoration” of the image or “deconvolution”. A restoration filter based on the point spread function (PSF) is generated considering information about the deterioration of the image during imaging, for example, imaging conditions (for example, an exposure time, the amount of exposure, the distance to the object, and a focal length) or characteristics information about an imaging device (for example, the optical characteristics of a lens and imaging device identification information) (JP2008-172321A).
A deterioration model due to blurring can be represented by a function. For example, a blurring phenomenon can be represented by a normal distribution having the distance (image height) from a central pixel as a parameter (JP2000-020691A).
JP2009-165115A discloses an example of an imaging device in which 3×3 (=9) pixels on an imaging element are allocated to one microlens.